Endoscopy support apparatus, endoscopy support method, and computer readable recording medium

ABSTRACT

An endoscopy support apparatus 1 includes: an analysis information generation unit 2 that inputs image information of an imaged living body into a model, estimates a region of a target site of the living body, and generates analysis information including region information indicating the estimated region and score information indicating likeness of the region to the target site; a user information generation unit 3 that generates user information related to the user, which has been input by a user using a user interface 23; an image-related information generation unit 4 that generates image-related information by associating imaging date and time information indicating a date and time when an image was captured, the analysis information, and the user information, for each piece of image information; and an examination management information generation unit 5 that generates examination management information by associating a plurality of pieces of the image-related information with examination information indicating an examination period.

TECHNICAL FIELD

The present invention relates to an endoscopy support apparatus and anendoscopy support method that support endoscopy, and further relates toa computer readable recording medium on which a program for realizingthem is recorded.

BACKGROUND ART

When a user performs an examination, an endoscopy system obtains a largeamount of image information captured by an endoscope, performs imageanalysis on each obtained piece of image information, and uses the imageinformation and image analysis results to present a living body internalimage and the image analysis results to the user. However, in theendoscopy system, the living body internal image and the image analysisresults can be displayed on a display of a display device in real timeduring the examination, but a large amount of image information and theimage analysis results are not organized and are stored in a storagedevice in a complicated manner.

As a related technique, Patent Document 1 discloses a medical image dataprocessing system for efficiently using medical image data. According tothe medical image data processing system of Patent Document 1, themedical image data is managed by storing the medical image data andmanagement information indicating the contents of the medical imagedata. Note that the management information is, for example, a managementnumber of the medical image data, a modality type, imaging date andtime, a patient name, a user name (a doctor name or an imagingengineer), an imaging site, imaging conditions, a reference imagenumber, and the like.

LIST OF RELATED ART DOCUMENTS Patent Document

Patent Document 1: Japanese Patent Laid-Open Publication No. 2008-079648

SUMMARY Technical Problems

However, the medical image data processing system of Patent Document 1is not a system that organizes and manages a large amount of obtainedimage information for each examination. Further, the medical image dataprocessing system of Patent Document 1 is not an apparatus that performsimage analysis on each piece of a large amount of obtained imageinformation.

Furthermore, since the conventional endoscopy system is assumed to beused continuously by a plurality of users, it is desired that a largenumber of living body internal images obtained during an examinationperiod and the image analysis results are organized for each examinationso that the user can efficiently use them.

An example object of the invention is to provide an endoscopy supportapparatus and an endoscopy support method that organize a large numberof biological images obtained during the examination period and theimage analysis results for each examination, and a computer readablerecording medium.

Solution to the Problems

In order to achieve the above object, an endoscopy support apparatus inone aspect of the present invention includes:

an analysis information generation means for inputting image informationof an imaged living body into a model, estimating a region of a targetsite of the living body, and generating analysis information includingregion information indicating the estimated region and score informationindicating likeness of the region to the target site;

a user information generation means for generating user informationrelated to a user, which has been input by the user using a userinterface;

an image-related information generation means for generatingimage-related information by associating imaging date and timeinformation indicating a date and time when an image was captured, theanalysis information, and the user information, for each piece of imageinformation; and

an examination management information generation means for generatingexamination management information by associating a plurality of piecesof the image-related information generated during an examination periodwith examination information indicating examination.

Further, in order to achieve the above object, an endoscopy supportmethod in one aspect of the present invention includes:

inputting image information of an imaged living body into a model,estimating a region of a target site of the living body, and generatinganalysis information including region information indicating theestimated region and score information indicating likeness of the regionto the target site;

generating user information related to a user, which has been input bythe user using a user interface;

generating image-related information by associating imaging date andtime information indicating a date and time when an image was captured,the analysis information, and the user information, for each piece ofimage information; and

generating examination management information by associating a pluralityof pieces of the image-related information with examination informationindicating an examination period.

Further, in order to achieve the above object, a computer readablerecording medium in one aspect of the present invention includes aprogram recorded thereon, the program including instructions that causea computer to carry out:

inputting image information of an imaged living body into a model,estimating a region of a target site of the living body, and generatinganalysis information including region information indicating theestimated region and score information indicating likeness of the regionto the target site;

generating user information related to a user, which has been input bythe user using a user interface;

generating image-related information by associating imaging date andtime information indicating a date and time when an image was captured,the analysis information, and the user information, for each piece ofimage information; and

generating examination management information by associating a pluralityof pieces of the image-related information with examination informationindicating an examination period.

Advantageous Effects of the Invention

As described above, according to the present invention, it is possibleto organize a large number of biological images obtained during theexamination period and the image analysis results, for each examination.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram for describing an example of an endoscopy supportapparatus.

FIG. 2 is a diagram for describing an example of a system including theendoscopy support apparatus.

FIG. 3 is a diagram for describing an example of a user settinginformation generation screen.

FIG. 4 is a diagram for describing an example of a profile selectionscreen and a profile screen.

FIG. 5 is a diagram for describing an example of a data structure ofimage-related information.

FIG. 6 is a diagram for describing an example of an examination screen.

FIG. 7 is a diagram for describing an example of a data structure ofexamination management information.

FIG. 8 is a diagram for describing an example of an examination resultscreen.

FIG. 9 is a diagram for describing an operation example of the endoscopysupport apparatus.

FIG. 10 is a diagram for describing an operation example of theendoscopy support apparatus.

FIG. 11 is a diagram illustrating an example of a computer for realizingthe endoscopy support apparatus.

EXAMPLE EMBODIMENT Example Embodiment

Hereinafter, an example embodiment of the present invention will bedescribed with reference to FIGS. 1 to 11.

[Apparatus Configuration]

First, a configuration of an endoscopy support apparatus in the presentexample embodiment will be described with reference to FIG. 1. FIG. 1 isa diagram for describing an example of the endoscopy support apparatus.

An endoscopy support apparatus 1 is an apparatus for organizing a largenumber of living body internal images obtained during an examinationperiod and image analysis results thereof. The endoscopy supportapparatus 1 is, for example, an information processing apparatus such asa personal computer or a server computer. As illustrated in FIG. 1, theendoscopy support apparatus 1 includes an analysis informationgeneration unit 2, a user information generation unit 3, animage-related information generation unit 4, and an examinationmanagement information generation unit 5.

The analysis information generation unit 2 inputs image information ofan imaged living body to a model, estimates a region of a target site ofthe living body, and generates analysis information including regioninformation indicating the estimated region, and score informationindicating likeness of the region to the target site.

The image information is information indicating, for example, abiological image of an internal or external organ of a living body, suchas a human, an animal, or a plant, captured by an imaging device mountedon an endoscope. The target site is, for example, a region with a lesionor an abnormality. The region information is a region estimated to bethe target site included in the biological image captured by theendoscope. The score information is an index indicating a degree (or aprobability) that the region estimated to be the target site isestimated to be the lesion or the abnormality.

The model is a model generated by artificial intelligence (AI), machinelearning, or the like. The model may be provided, for example, in theendoscopy support apparatus 1 or outside the endoscopy support apparatus1.

The user information generation unit 3 generates user informationrelated to a user, which is input by the user using a user interface.The image-related information generation unit 4 generates image-relatedinformation by associating imaging date and time information indicatinga date and time when the image was captured, the analysis information,and the user information, for each piece of image information. Theexamination management information generation unit 5 generatesexamination management information by associating a plurality of piecesof image-related information with examination information indicating theexamination period.

As described above, in the present example embodiment, since theimage-related information including the image information, the analysisinformation, and the user information can be managed for an examination,the user can efficiently use the image information and the analysisinformation after the examination.

[System Configuration]

Subsequently, the configuration of the endoscopy support apparatus 1 inthe present example embodiment will be described more specifically withreference to FIG. 2. FIG. 2 is a diagram for describing an example of asystem including the endoscopy support apparatus.

As illustrated in FIG. 2, a system 20 including the endoscopy supportapparatus 1 in the present example embodiment includes an endoscope 21,a control unit 22, a user interface 23, and an output device 24 inaddition to the endoscopy support apparatus 1. Further, the endoscopysupport apparatus 1 includes a user setting information generation unit6, an examination screen generation unit 7, and an examination resultscreen generation unit 8 in addition to the analysis informationgeneration unit 2, the user information generation unit 3, theimage-related information generation unit 4, and the examinationmanagement information generation unit 5.

The endoscope 21 transmits an image of the inside of the living body,which is obtained by imaging the inside of the living body, to thecontrol unit 22 connected to the endoscope 21. The endoscope 21includes, for example, an insertion portion to be inserted inside theliving body, an imaging device such as a camera provided on a tip sideof the insertion portion, an operating unit that operates curvature ofthe insertion portion, imaging of an imaging unit, and the like, and aconnection portion that connects the endoscope 21 and the endoscopysupport apparatus 1. Further, in addition to the imaging device, theendoscope 21 includes a lighting unit, a nozzle used for air supply,water supply, and suction, a forceps mouth, and the like on the tip sideof the insertion portion.

The control unit 22 is, for example, a video processor or the like thatperforms image processing or the like on the input image. Specifically,the control unit 22 obtains an imaging signal from the endoscope 21,performs image adjustment or the like on the imaging signal, generatesthe image obtained by imaging the inside of the living body, and outputsit to the endoscopy support apparatus 1. Note that the control unit 22may be provided in the endoscopy support apparatus 1.

The user interface 23 is an input screen or the like displayed on adisplay of the output device 24 when the user inputs information.Further, the user inputs information through the user interface 23 usingan operating device. Specifically, the user inputs information using avisual element (graphical information) displayed on the display, akeyboard, a mouse, a touch panel, or the like.

The output device 24 outputs images, sounds, and the like. The outputdevice 24 is, for example, an image display device including liquidcrystal, organic electroluminescence (EL), and a cathode ray tube (CRT).Further, the output device 24 includes an audio output device or thelike such as a speaker. Note that the output device 24 may be a printingdevice such as a printer.

The endoscopy support apparatus will be described in detail. The userinformation generation unit 3 generates the user information on thebasis of the information related to the user input by the user using theuser interface 23. The user information may include identificationinformation for identifying the user to be examined, gender informationindicating a gender of the user, age information indicating an age ofthe user, job title information indicating a job title of the user,experience information indicating an experience value and years ofexperience of the user, name recognition information indicating namerecognition of the user, evaluation information indicating evaluation ofthe user, and the like. However, the user information is only requiredto include the identification information and any one or more pieces ofinformation described above other than the identification information.

Specifically, first, the user information generation unit 3 displays theuser interface 23 for inputting the information related to the user onthe display of the output device 24. Subsequently, the user informationgeneration unit 3 generates the user information on the basis of theinformation related to the user, which has been input through the userinterface 23. Subsequently, the user information generation unit 3stores the user information in the storage device.

The user setting information generation unit 6 generates user settinginformation (a profile) used for setting output of the analysisinformation for each user. The user setting information includes atleast one of setting information for changing display of the regionaccording to the score information and setting information for changinga volume according to the score information, in the examination.

Specifically, first, the user setting information generation unit 6displays a user setting information generation screen as the userinterface on the display of the output device 24. Subsequently, the usersetting information generation unit 6 obtains the information input bythe user by using the user setting information generation screen.Subsequently, the user setting information generation unit 6 generatesthe user setting information using the obtained information, and storesthe user setting information in the storage device. The user settinginformation generation screen is, for example, a screen as illustratedin FIG. 3.

FIG. 3 is a diagram for describing an example of the user settinginformation generation screen. On a user setting information generationscreen 300 illustrated in FIG. 3, a threshold setting unit 301, adetection color setting unit 302, a detection sound setting unit 303,and an adjustment screen 304 are displayed.

The threshold setting unit 301 sets a threshold value used to divide ascore into a plurality of ranges. In the example of FIG. 3, there isprovided a control bar including a scale display 305 for setting athreshold value 1 and a scale display 306 for setting a threshold value2 in order to determine three score ranges of “HIGH”, “MIDDLE”, and“LOW”. In the example of FIG. 3, the user can change the threshold valueby moving the scale displays 305 and 306 of the control bar up and down.

The scale display 305 is used to determine a lower limit of the scorerange corresponding to “HIGH” or an upper limit of the score rangecorresponding to “MIDDLE”. The scale display 306 is used to determine alower limit of the score range corresponding to “MIDDLE” or an upperlimit of the score range corresponding to “LOW”. Further, the set scoresare displayed in the threshold values 1 and 2 in FIG. 3.

However, the score range is not limited to the three ranges. Further, inthe example of FIG. 3, the control bar is used for adjusting thethreshold values, but adjustment of the threshold values is not limitedto the control bar, and another input method may be employed.

The detection color setting unit 302 is used to set a color of adetection range display used to make the region of the detected targetsite easy for the user to understand. The detection color setting unit302 is used for setting to change the color of the detection rangedisplay according to the score corresponding to the region of thedetected target site when the region of the target site is detected.

The detection range display is displayed as 307, 308, and 309 on theadjustment screen 304 of FIG. 3. In FIG. 3, a difference in color isrepresented by a difference in pattern.

Further, in the example of FIG. 3, since the three score ranges of“HIGH”, “MIDDLE”, and “LOW” are set, a HIGH color setting display 310, aMIDDLE color setting display 311, and a LOW color setting display 312are used to set colors corresponding to the three score ranges of“HIGH”, “MIDDLE”, and “LOW”.

In the case of the HIGH color setting display 310, the color is set bycausing the user to select one of three color circular displays (1 to 3)displayed on the HIGH color setting display 310. Note that in FIG. 3,the difference in color is represented by the difference in pattern. Thecolors of the MIDDLE color setting display 311 and the LOW color settingdisplay 312 are also set as described above. However, the types of thecolors are not limited to three.

The detection range displays 307, 308, and 309 are donut-shaped on theadjustment screen 304 of FIG. 3, but are not limited to the donut shape.For example, the shape may extend along the region. That is, the displayis only required to be such that the region of the target site can beeasily understood by the user.

The detection sound setting unit 303 is used to set the volume used tomake it easy for the user to understand that the region of the targetsite has been detected. The detection sound setting unit 303 is used forsetting to change the volume according to the score corresponding to theregion of the detected target site when the region of the target site isdetected.

In the example of FIG. 3, since the three score ranges of “HIGH”,“MIDDLE”, and “LOW” are set, a HIGH sound setting display 313, a MIDDLEsound setting display 314, and a LOW sound setting display 315 are usedto set volumes corresponding to the three score ranges of “HIGH”,“MIDDLE”, and “LOW”.

In the case of the HIGH sound setting display 313, the volume is set bycausing the user to select one of five volume displays (1 to 5)displayed on the HIGH sound setting display 313 together with the soundcorresponding to HIGH. Note that in FIG. 3, the larger the number, thelouder the volume. The volumes of the MIDDLE sound setting display 314and the LOW sound setting display 315 are also set as described above.However, the types of volumes are not limited to five.

Further, a detection sound may be a sound that differs depending on thescore range. Further, the detection sound may be, for example, a soundsuch as music, a buzzer sound, or a voice.

The adjustment screen 304 is a screen referred to by the user when theabove-mentioned threshold setting, detection color setting, anddetection sound setting are performed. Specifically, the adjustmentscreen 304 is a screen used by the user to refer to an adjustment image,change each setting, and select a setting that is easy for the user touse.

The analysis information generation unit 2 analyzes the imageinformation of the imaged living body and generates the analysisinformation. The analysis information includes the region informationindicating the region of the target site (the target site image) and thescore information indicating the likeness to the target site.

Specifically, first, the analysis information generation unit 2 obtainsfrom the control unit 22 the image information generated by the controlunit 22 using the imaging signal output from the endoscope 21.Subsequently, the analysis information generation unit 2 inputs theimage information into the model and obtains the analysis information(the region information and the score information) output from themodel. Subsequently, the analysis information generation unit 2 outputsthe analysis information to the image-related information generationunit 4 and the examination screen generation unit 7.

When the image-related information generation unit 4 obtains startinformation indicating start of the examination, the image-relatedinformation generation unit 4 generates the image-related information byassociating the imaging date and time information, the analysisinformation, and the user information, for each piece of imageinformation. Note that information other than the imaging date and timeinformation, the analysis information, and the user information may beassociated with the image information.

FIG. 4 is a diagram for describing an example of a profile selectionscreen and a profile screen. For example, the examination screengeneration unit 7 displays on the display of the output device 24 aprofile selection screen 41 as illustrated in FIG. 4, as the userinterface. Subsequently, when an icon corresponding to the profile to beused is selected by the user, a profile screen 42 displaying contents ofa preset profile corresponding to the selected icon is displayed on thedisplay.

Specifically, when an icon “user_1” of the profile selection screen 41is selected by the user in FIG. 4, the preset profile screen 42corresponding to the icon “user_1” is displayed. Subsequently, when theuser determines that the examination is to be started with the contentsdisplayed on the profile screen 42, an examination start display 43 isselected by the user, and the start information is output to theimage-related information generation unit 4.

Note that the profile selection screen 41 may display an icon that canbe shared and used by the user. For example, an icon associated with aprofile of a well-known doctor, an icon associated with a profileproduced according to the race, gender, and age of a patient, and thelike are conceivable.

Subsequently, the image-related information generation unit 4 generatesthe image-related information by associating the imaging date and timeinformation, the analysis information (the region information and thescore information) corresponding to the image information output fromthe model, and the user information, for each piece of image informationinput to the model included in the analysis information generation unit2. The image-related information is, for example, informationillustrated in FIG. 5.

FIG. 5 is a diagram for describing an example of a data structure of theimage-related information. “Image_1”, “image_2”, “image_3”, . . . of the“image information” illustrated in FIG. 5 indicate, for example, filenames of the image information. “2019/11/10/11:20:34:120”,“2019/11/10/11:20:34:121”, “2019/11/10/11:20:34:122”, . . . of the“imaging date and time information” illustrated in FIG. 5 indicate, forexample, the date and time (year, month, day, time) when the image wascaptured. “Area_1”, “area_2”, “area_3”, . . . of the “regioninformation” in the “analysis information” illustrated in FIG. 5 are,for example, coordinates for indicating at least one region of a lesionor an abnormality, on the image. “Score_1”, “score_2”, “score_3”, . . .of the “score information” in the “analysis information” illustrated inFIG. 5 are, for example, indices indicating degrees (or probabilities)that a detected tumor region is a tumor region when a tumor is detectedas a lesion in the image.

Note that the image-related information generation unit 4 may furthergenerate the image-related information by associating order informationindicating the order in which the images were captured in theexamination with the image information.

Further, the image-related information may be associated with captureinformation indicating that the user captured the image using theendoscope. Thus, the user can easily view the captured image after theexamination. Note that examples of a capture detection method include amethod of detecting a preset number of the same image (or images withlittle change), a method of continuously detecting the same image for apreset period, and a method of detecting that a snapshot has beengenerated.

Further, the image-related information may be associated with adjustmenthistory information indicating that adjustment of the color and volumeof the detection range display performed by the user during theexamination was performed.

When the examination management information generation unit 5 obtainsend information indicating an end of the examination, the examinationmanagement information generation unit 5 generates the examinationmanagement information for managing the examination by associating theexamination information indicating the examination period with theplurality of pieces of image-related information generated during theexamination period.

Specifically, first, the examination management information generationunit 5 obtains the end information indicating the end of theexamination. FIG. 6 is a diagram for describing an example of anexamination screen. For example, the examination screen generation unit7 displays an examination screen 61 as illustrated in FIG. 6 as the userinterface on the display of the output device 24.

In the example of FIG. 6, the examination screen 61 displays an imagedisplay 62 for displaying the image of the living body and a detectionrange display 63 for displaying the region of the target site of theimage display 62. In addition, there are displayed a color adjustmentunit 64 for adjusting the color of the detection range display duringthe examination, a volume adjustment unit 65 for adjusting the volumeduring the examination, and an examination end display 66 for the userto select when the examination is completed.

When the user determines that the examination is completed, theexamination end display 66 is selected by the user, and the endinformation is output to the examination management informationgeneration unit 5. Subsequently, the examination management informationgeneration unit 5 generates the examination management information byassociating the plurality of pieces of image-related informationgenerated during the examination period with the examination informationindicating a period during which the examination is currently beingperformed. Subsequently, the examination management informationgeneration unit 5 stores the generated examination managementinformation in the storage device. The examination managementinformation is, for example, information as illustrated in FIG. 7.

FIG. 7 is a diagram for describing an example of a data structure of theexamination management information. “Check_1” of the “examinationinformation” illustrated in FIG. 7 is information for identifying theexamination performed by the user. In the example of FIG. 7, in theexamination of “check_1”, “image_XXXXXX” is obtained from “image_1” asthe image information, and the related information is associated witheach piece of image information obtained.

In addition, in FIG. 7, the region information and the score informationare associated with all of the image information, but when the targetsite is not detected, information is stored indicating that there is nodata in the region information and the score information.

Thus, the image-related information can be managed for each examination,and therefore the user can efficiently use the image information and theanalysis information after the examination.

The examination screen generation unit 7 generates output informationfor outputting the examination screen (the image, the sound, and thelike) to the output device 24 on the basis of the image information, theanalysis information, and the user setting information. Specifically,the examination screen generation unit 7 generates the outputinformation for displaying the examination screen 61 as illustrated inFIG. 6 on the display of the output device 24. Further, the examinationscreen generation unit 7 generates the output information for outputfrom the speaker of the output device 24.

Note that when the user is a doctor and shows and explains theexamination screen to the patient during the examination, in order tomake the color easy for the patient to understand, the user changes thecolor by using the color adjustment unit 64. Further, when the sound isnot required in the case of explanation, the volume adjustment unit 65is used to minimize the volume.

The examination result screen generation unit 8 obtains the examinationmanagement information from the storage device, and outputs the image,the sound, and the like to the output device 24. FIG. 8 is a diagram fordescribing an example of the examination result screen. The examinationresult screen generation unit 8 generates the output information fordisplaying, for example, an examination result screen 81 as illustratedin FIG. 8 on the display of the output device 24. Further, theexamination result screen generation unit 8 generates, for example, theoutput information for output from the speaker of the output device 24.

In the example of FIG. 8, the examination result screen 81 includes afilter unit 82 that classifies and displays the image and the analysisresult according to the score information associated with the imageinformation. Further, in the example of FIG. 8, since “MIDDLE” isselected (black circle) by the user in the filter unit 82, the imageassociated with the score included in the score range of “MIDDLE”, theimaging date and time, and the score are displayed in a relatedinformation display unit 83. Note that if “ALL” is selected, all of theimages can be referred to. Further, in the example of FIG. 8, anexamination result display unit 84 displays an examination result imagecorresponding to “01152” selected by the user by using the relatedinformation display unit 83.

Note that after the examination is completed, the adjustment made duringthe examination may be fed back to automatically change profilesettings. For example, the profile setting may be changed on the basisof the adjustment history information included in the image-relatedinformation described above.

[Apparatus Operation]

Next, an operation of the endoscopy support apparatus in an exampleembodiment of the present invention will be described with reference toFIGS. 9 and 10. FIGS. 9 and 10 are diagrams for describing an operationexample of the endoscopy support apparatus. In the followingdescription, FIGS. 1 to 8 will be referred to as appropriate. Further,in the present example embodiment, an endoscopy support method isimplemented by operating the endoscopy support apparatus. Therefore,description of the endoscopy support method in the present exampleembodiment is replaced with the following operation description of theendoscopy support apparatus.

Settings before the examination will be described.

As illustrated in FIG. 9, first, the user information generation unit 3generates the user information related to the user, which has been inputby the user using the user interface 23 (step A1).

Specifically, in step A1, first, the user information generation unit 3displays the user interface 23 for inputting the information related tothe user on the display of the output device 24. Subsequently, in stepA1, the user information generation unit 3 obtains the user informationinput through the user interface 23. Subsequently, in step A1, the userinformation generation unit 3 stores the user information in the storagedevice.

Subsequently, the user setting information generation unit 6 generatesthe user setting information (the profile) used for setting the outputof the analysis information for each user (step A2).

Specifically, in step A2, first, the user setting information generationunit 6 displays the user setting information generation screen as theuser interface on the display of the output device 24. Subsequently, instep A2, the user setting information generation unit 6 obtains theinformation input by the user using the user setting informationgeneration screen. Subsequently, in step A2, the user settinginformation generation unit 6 generates the user setting informationusing the obtained information, and stores the user setting informationin the storage device.

The operation during the examination will be described.

As illustrated in FIG. 10, first, the image-related informationgeneration unit 4 obtains the start information indicating the start ofthe examination (step B1).

Specifically, when the icon of the profile selection screen 41 displayedon the display of the output device 24 is selected by the user, theprofile corresponding to the selected icon is displayed on the profilescreen 42. Subsequently, when the user determines that the examinationis to be started with the contents displayed on the profile screen 42,the examination start display 43 is selected by the user, and the startinformation is output to the image-related information generation unit4.

Subsequently, the image-related information generation unit 4 generatesthe image-related information by associating the imaging date and timeinformation, the analysis information, the user information, and thelike, for each piece of image information (step B2).

Specifically, in step B2, the image-related information generation unit4 generates the image-related information by associating the imagingdate and time information indicating the date and time when the imagewas captured, the analysis information (the region information and thescore information) corresponding to the image information output fromthe model, and the user information, for each piece of image informationinput to the model included in the analysis information generation unit2. The image-related information is, for example, the informationillustrated in FIG. 5.

Note that the image-related information generation unit 4 may furthergenerate the image-related information by associating the orderinformation indicating the order in which the images were captured inthe examination with the image information.

Further, the image-related information may be associated with thecapture information indicating that the user captured the image usingthe endoscope. Thus, the user can easily view the captured image later.Note that examples of the capture detection method include a method ofdetecting that the same image (or images with little change) hascontinued for a preset number of times, a method of detecting that thesame image has continued for a preset period, and a method of detectingthat a snapshot has been taken.

Further, the image-related information may be associated with theadjustment history information indicating that the color and volume ofthe detection range display made by the user during the examination wereadjusted.

Subsequently, the examination screen generation unit 7 obtains the imageinformation, the analysis information, and the user setting information,and generates the output information for outputting the examinationscreen (the image, the sound, and the like) to the output device 24 onthe basis of the obtained image information, analysis information, anduser setting information (step B3).

Specifically, in step B3, the examination screen generation unit 7generates the output information for displaying the examination screen61 as illustrated in FIG. 6 on the display of the output device 24.Further, in step B3, the examination screen generation unit 7 generatesthe output information for output from the speaker of the output device24.

Note that when the user is the doctor and shows and explains theexamination screen to the patient during the examination, in order tomake the color easy for the patient to understand, the user changes thecolor by using the color adjustment unit 64. Further, when the sound isnot required in the case of explanation, the volume adjustment unit 65is used to minimize the volume.

Note that the order of processing in step B2 and step B3 may bereversed.

Subsequently, the examination management information generation unit 5determines whether or not the end information indicating the end of theexamination has been obtained (step B4). If the end information has beenobtained (step B4: Yes), the process proceeds to step B5. If the endinformation has not been obtained (step B4: No), the process proceeds tostep B2 and the processes of steps B2 and B3 are continued.

Specifically, in step B4, when the user determines that the examinationis completed, the examination end display 66 on the examination screen61 displayed on the display of the output device 24 is selected by theuser, and the end information is output to the examination managementinformation generation unit 5.

Subsequently, the examination management information generation unit 5generates the examination management information by associating theplurality of pieces of image-related information generated during theexamination period with the examination information indicating theperiod during which the examination is currently being performed (stepB5).

Specifically, in step B5, the examination management informationgeneration unit 5 generates the examination management information byassociating the plurality of pieces of image-related informationgenerated during the examination period with the examination informationindicating the period during which the examination is currently beingperformed. Subsequently, in step B5, the examination managementinformation generation unit 5 stores the generated examinationmanagement information in the storage device.

When the examination result is displayed on the output device 24 afterthe examination is completed, the examination result screen generationunit 8 obtains the examination management information from the storagedevice and outputs the image, the sound, or the like to the outputdevice 24. Specifically, the examination result screen generation unit 8generates the output information for displaying the examination resultscreen 81 as illustrated in FIG. 8 on the display of the output device24. Further, the examination result screen generation unit 8 generates,for example, the output information for output from the speaker of theoutput device 24.

Note that after the examination is completed, the adjustment made duringthe examination may be fed back to automatically change the profilesettings. For example, the profile setting may be changed on the basisof the adjustment history information included in the image-relatedinformation described above.

Effect of the Present Example Embodiment

As described above, according to the present example embodiment, theimage-related information including the image information, the analysisinformation, and the user information can be managed for eachexamination, and therefore the user can efficiently use the imageinformation and the analysis information after the examination.

[Program]

A program in the example embodiment of the present invention may be anyprogram that causes a computer to execute steps A1 and A2 illustrated inFIG. 9 and steps B1 to B5 illustrated in FIG. 10. By installing andexecuting the program on the computer, the endoscopy support apparatusand the endoscopy support method in the present example embodiment canbe implemented. In this case, a processor of the computer functions asthe analysis information generation unit 2, the user informationgeneration unit 3, the image-related information generation unit 4, theexamination management information generation unit 5, the user settinginformation generation unit 6, the examination screen generation unit 7,and the examination result screen generation unit 8, and performsprocessing.

Further, the program in the present example embodiment may be executedby a computer system constructed by a plurality of computers. In thiscase, for example, each computer may function as any of the analysisinformation generation unit 2, the user information generation unit 3,the image-related information generation unit 4, the examinationmanagement information generation unit 5, the user setting informationgeneration unit 6, the examination screen generation unit 7, and theexamination result screen generation unit 8.

[Physical Configuration]

Here, a computer for implementing the endoscopy support apparatus byexecuting the program in the example embodiment will be described withreference to FIG. 11. FIG. 11 is a block diagram illustrating an exampleof the computer for implementing the endoscopy support apparatus in theexample embodiment of the present invention.

As illustrated in FIG. 11, a computer 110 includes a central processingunit (CPU) 111, a main memory 112, a storage device 113, an inputinterface 114, a display controller 115, a data reader/writer 116, and acommunication interface 117. These parts are connected to each other viaa bus 121 so as to be capable of data communication. Note that thecomputer 110 may include a graphics processing unit (GPU) or afield-programmable gate array (an FPGA) in addition to the CPU 111 or inplace of the CPU 111.

The CPU 111 expands the program (code) in the present example embodimentstored in the storage device 113 into the main memory 112 and executesthe program in a predetermined order to perform various operations. Themain memory 112 is typically a volatile storage device such as a dynamicrandom access memory (DRAM). Further, the program in the present exampleembodiment is provided in a state of being stored in a computer-readablerecording medium 120. The program in the present example embodiment maybe distributed on the Internet connected via the communication interface117. Note that the recording medium 120 is a non-volatile recordingmedium.

Further, examples of the storage device 113 include a semiconductorstorage device such as a flash memory in addition to a hard disk drive.The input interface 114 mediates data transmission between the CPU 111and an input device 118 such as a keyboard and a mouse. The displaycontroller 115 is connected to a display device 119 and controls displayon the display device 119.

The data reader/writer 116 mediates the data transmission between theCPU 111 and the recording medium 120, reads the program from therecording medium 120, and writes a processing result in the computer 110to the recording medium 120. The communication interface 117 mediatesthe data transmission between the CPU 111 and another computer.

Further, examples of the recording medium 120 include a general-purposesemiconductor storage device such as a compact flash (CF) (registeredtrademark) and a secure digital (SD), a magnetic recording medium suchas a flexible disk, or an optical recording medium such as a compactdisk read only memory (CD-ROM).

Note that the endoscope examination apparatus 1 in the present exampleembodiment can also be implemented by using hardware corresponding toeach part instead of the computer in which the program is installed.Further, the endoscope examination apparatus 1 may be partiallyimplemented by a program and the rest may be implemented by hardware.

[Supplementary Note]

Regarding the above example embodiment, the following supplementarynotes will be further disclosed. A part or all of the above-describedexample embodiment can be described by (Supplementary note 1) to(Supplementary note 12) described below, but it is not limited to thefollowing descriptions.

(Supplementary Note 1)

An endoscopy support apparatus including:

an analysis information generation unit configured to input imageinformation of an imaged living body into a model, estimate a region ofa target site of the living body, and generate analysis informationincluding region information indicating the estimated region and scoreinformation indicating likeness of the region to the target site;

a user information generation unit configured to generate userinformation related to a user, which has been input by the user using auser interface;

an image-related information generation unit configured to generateimage-related information by associating imaging date and timeinformation indicating a date and time when an image was captured, theanalysis information, and the user information, for each piece of imageinformation; and

an examination management information generation unit configured togenerate examination management information by associating a pluralityof pieces of the image-related information with examination informationindicating an examination period.

(Supplementary Note 2)

The endoscopy support apparatus described in supplementary note 1,including a user setting information generation unit configured togenerate user setting information used for setting output of theanalysis information for each user.

(Supplementary Note 3)

The endoscopy support apparatus described in supplementary note 2, inwhich the user setting information includes at least one of settinginformation for changing display of the region according to the scoreinformation and setting information for changing a volume according tothe score information, in an examination.

(Supplementary Note 4)

The endoscopy support apparatus described in any one of supplementarynotes 1 to 3, in which when the examination management informationgeneration unit obtains end information indicating an end of theexamination, the examination management information generation unitgenerates the examination management information.

(Supplementary Note 5)

An endoscopy support method including:

inputting image information of an imaged living body into a model,estimating a region of a target site of the living body, and generatinganalysis information including region information indicating theestimated region and score information indicating likeness of the regionto the target site;

generating user information related to a user, which has been input bythe user using a user interface;

generating image-related information by associating imaging date andtime information indicating a date and time when an image was captured,the analysis information, and the user information, for each piece ofimage information; and

generating examination management information by associating a pluralityof pieces of the image-related information with examination informationindicating an examination period.

(Supplementary Note 6)

The endoscopy support method described in supplementary note 5,including generating user setting information used for setting output ofthe analysis information for each user.

(Supplementary Note 7)

The endoscopy support method described in supplementary note 6, in whichthe user setting information includes at least one of settinginformation for changing display of the region according to the scoreinformation and setting information for changing a volume according tothe score information, in an examination.

(Supplementary Note 8)

The endoscopy support method described in any one of supplementary notes5 to 7, in which when end information indicating an end of theexamination is obtained, the examination management information isgenerated.

(Supplementary Note 9)

A computer readable recording medium including a program recordedthereon, the program including instructions for causing a computer tocarry out:

inputting image information of an imaged living body into a model,estimating a region of a target site of the living body, and generatinganalysis information including region information indicating theestimated region and score information indicating likeness of the regionto the target site;

generating user information related to a user, which has been input bythe user using a user interface;

generating image-related information by associating imaging date andtime information indicating a date and time when an image was captured,the analysis information, and the user information, for each piece ofimage information; and

generating examination management information by associating a pluralityof pieces of the image-related information with examination informationindicating an examination period.

(Supplementary Note 10)

The computer readable recording medium described in supplementary note 9including a program recorded thereon, the program including instructionsthat cause a computer to carry out generating user setting informationused for setting output of the analysis information for each user.

(Supplementary Note 11)

The computer readable recording medium described in Supplementary note10, in which the user setting information includes at least one ofsetting information for changing display of the region according to thescore information and setting information for changing a volumeaccording to the score information, in an examination.

(Supplementary Note 12)

The computer readable recording medium described in any one ofSupplementary notes 9 to 11, in which when end information indicating anend of the examination is obtained, the examination managementinformation is generated.

Although the present invention has been described above with referenceto the example embodiment, the present invention is not limited to theabove example embodiment. Various changes that can be understood bythose skilled in the art can be made within the scope of the presentinvention in terms of the structure and details of the presentinvention.

INDUSTRIAL APPLICABILITY

As described above, according to the present invention, it is possibleto organize a large number of living body internal images obtainedduring the examination period and their image analysis results, for eachexamination. The present invention is useful in fields where managementof a large number of images is required.

REFERENCE SIGNS LIST

1 Endoscopy support apparatus

2 Analysis information generation unit

3 User information generation unit

4 Image-related information generation unit

5 Examination management information generation unit

6 User setting information generation unit

7 Examination screen generation unit

8 Examination result screen generation unit

20 System

21 Endoscope

22 Control unit

23 User interface

24 Output device

110 Computer

111 CPU

112 Main memory

113 Storage device

114 Input interface

115 Display controller

116 Data reader/writer

117 Communication interface

118 Input device

119 Display device

120 Recording medium

121 Bus

What is claimed is:
 1. An endoscopy support apparatus comprising: ananalysis information generation unit that inputs image information of animaged living body into a model, estimates a region of a target site ofthe living body, and generates analysis information including regioninformation indicating the estimated region and score informationindicating likeness of the region to the target site; a user informationgeneration unit that generates user information related to a user, whichhas been input by the user using a user interface; an image-relatedinformation generation unit that generates image-related information byassociating imaging date and time information indicating a date and timewhen an image was captured, the analysis information, and the userinformation, for each piece of image information; and an examinationmanagement information generation unit that generates examinationmanagement information by associating a plurality of pieces of theimage-related information generated during an examination period withexamination information indicating examination.
 2. The endoscopy supportapparatus according to claim 1, further comprising a user settinginformation generation unit that generates user setting information usedfor setting output of the analysis information for each user.
 3. Theendoscopy support apparatus according to claim 2, wherein the usersetting information includes at least one of setting information forchanging display of the region according to the score information andsetting information for changing a volume according to the scoreinformation, in an examination.
 4. The endoscopy support apparatusaccording to claim 1, wherein when the examination managementinformation generation unit obtains end information indicating an end ofthe examination, the examination management information generation unitgenerates the examination management information.
 5. An endoscopysupport method comprising: inputting image information of an imagedliving body into a model, estimating a region of a target site of theliving body, and generating analysis information including regioninformation indicating the estimated region and score informationindicating likeness of the region to the target site; generating userinformation related to a user, which has been input by the user using auser interface; generating image-related information by associatingimaging date and time information indicating a date and time when animage was captured, the analysis information, and the user information,for each piece of image information; and generating examinationmanagement information by associating a plurality of pieces of theimage-related information with examination information indicating anexamination period.
 6. The endoscopy support method according to claim5, further comprising generating user setting information used forsetting output of the analysis information for each user.
 7. Theendoscopy support method according to claim 6, wherein the user settinginformation includes at least one of setting information for changingdisplay of the region according to the score information and settinginformation for changing a volume according to the score information, inan examination.
 8. The endoscopy support method according to claim 5,wherein when end information indicating an end of the examination isobtained, the examination management information is generated.
 9. Anon-transitory computer readable recording medium including a programrecorded thereon, the program including instructions that cause acomputer to carry out: inputting image information of an imaged livingbody into a model, estimating a region of a target site of the livingbody, and generating analysis information including region informationindicating the estimated region and score information indicatinglikeness of the region to the target site; generating user informationrelated to a user, which has been input by the user using a userinterface; generating image-related information by associating imagingdate and time information indicating a date and time when an image wascaptured, the analysis information, and the user information, for eachpiece of image information; and generating examination managementinformation by associating a plurality of pieces of the image-relatedinformation with examination information indicating an examinationperiod.
 10. The non-transitory computer readable recording mediumaccording to claim 9 including a program recorded thereon, the programincluding instructions that cause a computer to carry out generatinguser setting information used for setting output of the analysisinformation for each user.
 11. The non-transitory computer readablerecording medium according to claim 10, wherein the user settinginformation includes at least one of setting information for changingdisplay of the region according to the score information and settinginformation for changing a volume according to the score information, inan examination.
 12. The non-transitory computer readable recordingmedium according to claim 9, wherein when end information indicating anend of the examination is obtained, the examination managementinformation is generated.